Method and system for automatically generating a section in a radiology report

ABSTRACT

A system  100  for automatically generating a field of a radiology report includes a set of one or more models. A method for automatically generating a field of a radiology report includes: receiving a radiologist identifier (radiologist ID); receiving a set of finding inputs; determining a context of each of the set of finding inputs; determining text associated with a portion or all of the radiology report based on the context and the radiologist style; and inserting the text into the report.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.17/020,593, filed 14 Sep. 2020, which claims the benefit of U.S.Provisional Application No. 62/900,148, filed 13 Sep. 2019, which isincorporated in its entirety by this reference.

TECHNICAL FIELD

This invention relates generally to the radiology field, and morespecifically to a new and useful system and method for the automatedgeneration of one or more sections of a radiology report in theradiology field.

BACKGROUND

Current radiology workflows are typically long and inefficient,requiring the radiologist to spend time and effort generating numerousfields within a radiology report. While automation of one or more fieldscan be implemented, the content and look of one or more fields of thereport, such as the impression field, are typically highly dependent onthe particular radiologist generating the report. Conventional attemptsto automate these fields leave most radiologists dissatisfied with theresults.

Thus, there is a need in the radiology field to create an improvedsystem and method for automatically generating one or more fields of aradiology report in an accurate way which is also satisfactory to theparticular radiologist generating the report.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 is a schematic of a system for the automated generation ofimpression text in a radiology report.

FIG. 2 is a schematic of a method for the automated generation ofimpression text in a radiology report.

FIGS. 3A-3D depict a variation of a method for the automated generationof impression text in a radiology report.

FIG. 4 depicts a schematic variation of a system for the automatedgeneration of impression text in a radiology report.

FIG. 5 depicts a schematic variation of the inputs and outputs in asystem and/or method for the automated generation of impression text ina radiology report.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

The following description of the preferred embodiments of the inventionis not intended to limit the invention to these preferred embodiments,but rather to enable any person skilled in the art to make and use thisinvention.

1. Overview

As shown in FIG. 1 , a system 100 for automatically generating a field(e.g., impression field) of a radiology report includes a set of one ormore models 110. One or more of the set of models preferably includes aset of encoders and a set of decoders implemented in a machine learningmodel, such as a transformer machine learning model (e.g.,multi-transformer model). Additionally or alternatively, the system caninclude and/or interface with any or all: of a pre-processing module112, a post-processing module 114, a computing system 120 and/orprocessing system, a radiology platform such as one including a speechrecognition platform, a set of devices (e.g., user devices), and/or anyother suitable components or combination of components.

Further additionally or alternatively, the system 100 can include and/orinterface with any or all of the systems, components, embodiments,and/or examples described in U.S. application Ser. No. 16/688,623, filed19 Nov. 2019, which is incorporated herein in its entirety by thisreference.

As shown in FIG. 2 , a method 200 for automatically generatingimpression text (and/or any other suitable fields of a radiology reportsuch as comparisons, contrast amounts, specific measurements, etc.)includes: receiving a radiologist identifier (radiologist ID); receivinga set of finding inputs and optionally other inputs; determining acontext of each of the set of inputs; determining an impression based onthe context and the radiologist style; and inserting the impression text(and/or any other suitable text) into the report. Additionally oralternatively, the method 200 can include any or all of: determining aradiologist style, training a set of models, preprocessing information,postprocessing information, and/or any other suitable processesperformed in any suitable order.

Further additionally or alternatively, the method 200 can include and/orinterface with any or all of the methods, processes, embodiments, and/orexamples described in U.S. application Ser. No. 16/688,623, filed 19Nov. 2019, which is incorporated herein in its entirety by thisreference.

The method 200 can be performed with a system as described above and/orany other suitable system.

2. Benefits

The system and/or method can confer several benefits over currentsystems and methods.

In a first variation, the system and/or method confers the benefit ofdecreasing the time and/or effort required for a radiologist to generatea report by automatically generating an impression from at least a setof findings.

In a second variation, additional or alternative to those describedabove, the system and/or method confers the benefit of mimicking aradiologist's writing style (e.g., word choice, grammar, consolidationand summary of findings, style of conclusions drawn from summarizedfindings, preferred follow-up recommendations, etc.) in the automatedgeneration of one or more sections of a radiology report. In a specificexample, the method confers the benefit of high user adoption andsatisfaction by producing an automated impression which closely mimicsand/or matches an impression which would have been manually generated bythe radiologist.

In a third variation, additional or alternative to those describedabove, the system and/or method confers the benefit of partially orfully standardizing any or all of the formatting and/or language ofreports generated for a particular radiology group and/or healthcarefacility, and of improving report recommendation adherence to consensusguidelines, billing and coding requirements, and/or quality metricsstandards. In specific examples, for instance, one or more automatedrecommendations in an automated impression section of the report areconfigured to adhere to any or all of a set of preferences,requirements, standards, and/or guidelines.

In a fourth variation, additional or alternative to those describedabove, the system and/or method confers the benefit of enabling azero-click generation of impressions without prematurely generating theimpression.

Additionally or alternatively, the system and method can confer anyother benefit.

3. System

The system 100 for the automated generation of a radiology impression(and/or any other suitable fields of a radiology report) includes a setof one or more models 110. Additionally or alternatively, the system caninclude and/or interface with any or all: of a pre-processing module112, a post-processing module 114, a computing system 120, a radiologyplatform, and/or any other suitable components or combination ofcomponents.

The system 100 preferably functions to generate one or more fields of aradiology report, wherein the generated fields preferably include animpressions section of a radiology report. The system 100 furtherpreferably functions to generate an impressions section which is similarto (e.g., imitates, mimics, replicates, approximates, etc.) in styleand/or content the impressions that the radiologist manually generatesand/or desires to generate.

Additionally or alternatively, the system 100 can function to generatean entire radiology report; facilitate compliance of a radiology reportwith one or more radiology standards and/or conventions (e.g.,recommended language); increase and/or maintain an accuracy above apredetermined threshold of the generated impression; optimize aradiology report for healthcare facility billing and/or reimbursements;and/or perform any other functions.

3.1 System—Set of Models

The system 100 includes a set of models 110, which individually and/orcollectively function to generate one or more fields of a radiologyreport which approximate the style of any or all of: a particularradiologist; an aggregated set of radiologists; an optimal standard forthat field; and/or any other suitable entities. Additionally oralternatively, the set of models no can function to perform any or allof: decreasing a processing time and/or improving one or more processingparameters (e.g., increasing accuracy, increasing radiologistapproval/satisfaction, decreasing radiologist edits to the generatedfields, increasing detail of a generated radiology impression,increasing readability and/or interpretability of a generated field,etc.); increasing a quality or consistency of a radiology report; and/orperforming any other suitable function(s).

The set of models are preferably located at (e.g., stored at, processedat, etc.) a computing system (e.g., as described above), furtherpreferably a remote computing system (e.g., cloud computing system,remote server, etc.), but can additionally or alternatively be locatedat any or all of: a local computing system, a combination of computingsystems, and/or at any other suitable location(s).

The set of models preferably includes one or more machine learningmodels, further preferably one or more deep learning models.Additionally or alternatively, the set of models can include any or allof: algorithms, equations, rules and/or rulesets, databases, lookuptables, and/or any other suitable tools for generating, checking,editing, and/or otherwise processing language information in a radiologyreport.

The set of models receives a set of inputs, wherein the set of inputspreferably includes at least a radiologist ID (e.g., as describedbelow), which functions to specify a particular radiologist's style withwhich to generate a radiology report field (e.g., impressions section),and a set of finding inputs (e.g., as described below), such as thecontents (e.g., sentences) of a findings field of the radiologistreport. The set of inputs further preferably includes a clinicalindication(s) section of the radiology report and optionally a reporttitle, technique, and/or set of comparison studies. Further additionallyor alternatively, the set of inputs can include any other sections ofthe report, other radiologist information, other patient information(e.g., from the radiology report, from outside of the radiology report,from a historical radiology report, etc.), healthcare facilityinformation, database information (e.g., from an EHR database, from anEMR database, from a PACS database, from a RIS database, etc.),radiology group information, radiology standards, billing proceduresand/or guidelines, and/or any other information from any suitablesources.

The set of finding inputs are preferably received from a speechrecognition platform, wherein the radiologist verbally dictatesinformation (e.g., the set of findings, information which is used todetermine the set of findings, etc.) which is transcribed by the speechrecognition platform into text. Additionally or alternatively, the setof finding inputs can be any or all of: received from the radiologyreport (e.g., already transcribed from a speech recognition platform,typed and/or written by a radiologist, etc.); received from a database,storage, and/or server; received as an output from a model (e.g.,predicted with a machine learning model); and/or otherwise determinedand/or received. The radiologist ID is preferably associated with (e.g.,references and/or retrieves from remote storage, from local storage,from a database, from a remote server, from a cloud computing system,etc.) a radiologist style matrix and/or vector (e.g., as describedbelow) which within the model(s) to help produce an impression whichmimics the style of the particular radiologist. The radiologist stylematrix is preferably part of the architecture of one or more models(e.g., as described below), but can additionally or alternatively beused as an input into one or more models (e.g., retrieved from a lookuptable based on the radiologist ID and used as an input to a transformermodel). Further additionally or alternatively, the radiologist style canbe represented in any other suitable form (e.g., set of weights, set ofparameters, etc.), include any suitable information (e.g., aggregatedstyle of a set of radiologists, optimal style of a theoreticalradiologist, optimal style for a radiology group and/or healthcarefacility and/or compliance with a set of radiology standards, etc.),and/or used in the model(s) in any suitable ways.

Additionally or alternatively, the set of models can receive anysuitable information as inputs, such as any or all of: any informationfrom a radiology report (e.g., the entire radiology report, any sectionfrom the radiology report, patient metadata, radiologist information,healthcare facility information, radiology group information, etc.); anyinformation from other radiology reports (e.g., prior report(s) of thepatient); information from a database, storage, server, and/or softwaretools (e.g., EMR database, EHR database, RIS, CIS, PACS, etc.); a set ofimages (e.g., diagnostic images of the patient); video; and/or anysuitable information.

The set of models preferably includes one or more neural networks (e.g.,feedforward neural networks, recurrent neural networks, convolutionalneural networks, etc.), but can additionally or alternatively includeany suitable algorithms (e.g., machine learning algorithms), decisiontrees, models, and/or any other suitable processing tools. The modelscan be trained through supervised learning (e.g., based on annotatedreports, based on manually generated reports, based on synthesizedreports, etc.), trained through unsupervised learning, untrained, orotherwise determined.

The set of models further preferably includes one or more deep learningmodels configured for natural language processing (NLP) (e.g., modelsconfigured to handle sequential data), such as one or more deep learningmodels with attention mechanisms, such as any or all of: asequence-to-sequence architecture; one or more attention layers (e.g.,in one or more encoders, in one or more decoders, etc.); one or moreself-attention layers (e.g., in one or more encoders, in one or moredecoders, etc.); and/or any other tools, features, and/or architecture.Additionally or alternatively, the deep learning model(s) can beconfigured for any suitable applications and/or otherwise designed.

The set of models preferably includes models implementingparallelization (e.g., processing all tokens at the same time) whereinprocessing data in order is not required, which can function to reducetraining times and processing times. In preferred variations, forinstance, the set of models includes a set of one or more transformers.In specific examples, for instance, the set of models includes atransformer model including multiple encoders with one or more decodersthat each consult one or more of the encoders sequentially. In aparticular example, the set of models includes a transformer model(equivalently referred to herein as a multi-transformer model) withmultiple decoders that each consult a set of multiple encoders in asequential fashion. Additionally or alternatively, the set of models caninclude any other suitable transformers and/or transformer systems(e.g., Bidirectional Encoder Representations from Transformers [BERT],Generative Pre-Trained Transformer [GPT], etc.); a transformer with anysuitable number and/or arrangement of encoders and decoders (e.g., equalnumber of encoders and decoders, more encoders than decoders, moredecoders than encoders, each decoder consulting one or more of theencoders in varying order, a single encoder, a single decoder, etc.); asingle transformer; multiple transformers; and/or any other suitabletransformers or models.

Additionally or alternatively, the set of models can include other NLPmodels such as recurrent neural networks (RNNs) (e.g., long short-termmemory [LSTM] models, gated recurrent units [GRUs], etc.) and/or anyother suitable models.

The model(s) can be any or all of: trained, pretrained, fine-tuned orusing other forms of transfer learning (e.g., based on a pretrainedmodel), combined with one or more ontologies (e.g., radiological orother clinical ontology database), and/or any combination of these. Insome variations, for instance, the set of models includes one or moretrained and/or pretrained models which are fine-tuned based on radiologyreport language.

The set of models 110 can optionally include and/or interface with apre-processing module 112, which functions to clean up and/or otherwisemodify data prior to training on and/or processing it. Thepre-processing module 112 is preferably implemented prior to or duringthe training of the model(s), but can additionally or alternatively beimplemented on data serving as input to the trained model, and/or beimplemented at any suitable time(s) during the method 200 in anysuitable way(s).

The set of models 110 can optionally include and/or interface with apost-processing module 114, which functions to edit and/or otherwisemodify one or more outputs produced by the set of models. This caninclude any or all of: formatting an output (e.g., an impressionssection); further improving language styling to better match the styleof the radiologist; checking and/or adjusting language for compliancewith recommended and/or required language (e.g., medical classificationlists such as the International Classification of Diseases and RelatedHealth Problems [ICD], ICD-10, usage of word “indicates” for diagnosesto conform with billing guidelines and/or requirements, merit-basedincentive payment system [MIPS] to help with and/or maximizereimbursement, etc.); notifying the radiologist of language whichpotentially may not conform with recommended and/or required language(e.g., as described above, so that the radiologist may manually edit,etc.); and/or any other processing. Additionally or alternatively, anyor all of the above can be performed in pre-processing, with the set ofmodels 110, and/or at any suitable time(s) during the method 200 withany suitable models and/or modules.

Any or all of the set of models 110, pre-processing module, andpost-processing module can be configured according to the type ofimaging (e.g., X-ray, MRI, CT, ultrasound, etc.) associated with theradiology report. The type of imaging, for instance, can be associatedwith any or all of: different report templates that need to be removedor modified in pre-processing; different styles and/or content ofimpressions (e.g., no further imaging recommendations for certainfindings on MRI reports because it is already the gold standard forimaging those findings, different conditions and/or pathologies beinginvestigated, etc.); different formatting; and/or any other differences.Additionally or alternatively, the models can be otherwise configuredbased on any other suitable features and/or information.

In a first variation, the set of models includes one or moretransformers (e.g., multi-transformer), which receive at least aradiologist ID and/or radiologist style matrix, along with a transcribedfindings section of a radiologist report as inputs, wherein the one ormore transformers produce an impressions section which mimics theradiologist's style and accurately assesses the findings. The set ofmodels further preferably receives as an input a clinical indication(s)section from the radiology report, and optionally any or all of: thereport title, technique section (e.g., how the exam was performed andwhether contrast was used in the imaging), comparison information (e.g.,comparison section of report, list of comparison studies, etc.), and/orany other suitable information.

In specific examples (e.g., as shown in FIG. 4 ), the system 100includes at least one transformer model, wherein the transformer modelincludes a set of encoders and a set of decoders (e.g., beam searchdecoders) configured to determine a string of impression text based onat least a radiologist ID and one or more sections (e.g., findingssection) of a radiology report.

3.2 System—Computing System 120

The system 100 can include and/or interface with a computing system 120,which functions to implement the set of models. Additionally oralternatively, the computing system can function to store one or moremodels and/or information, train one or more models, pre-processinformation, post-process information, and/or perform any other suitablefunctions.

The computing system can include and/or interface with any or all of: aprocessing system (e.g., set of processors, set of microprocessors,CPUs, GPUs, etc.), storage, memory, software programs/tools (e.g.,radiology software, a speech recognition platform, etc.), and/or anyother suitable components.

The computing system is preferably at least partially arranged remotely,but can additionally or alternatively be arranged locally (e.g., at ahealthcare facility, at a radiologist's workstation, etc.), at a device(e.g., mobile device, stationary device, etc.) and/or among multipledevices, and/or at any suitable locations.

In a first variation, the set of models 110 are processed at a remotecomputing system, wherein the remote computing system interfaces with aradiology platform (e.g., as described below), wherein the computingsystem receives one or more of a set of inputs from the radiologyplatform (e.g., findings) and transmits a generated impression to theradiology platform to be integrated into the radiology report.

3.3 System—Radiology Platform

The system 100 is preferably configured to interface with a radiologyplatform (e.g., radiology reporting platform, PowerScribe, Fluency forImaging, etc.), wherein the radiology platform preferably includesand/or interfaces with a speech recognition system which is equivalentlyreferred to herein as any or all of: a speech recognition platform, aspeech transcription system and/or platform, a voice recognition systemand/or platform, a voice transcription system and/or platform, aspeech-to-text system and/or platform, and/or any other suitableplatform including any suitable tools and/or programs.

Additionally or alternatively, the system 100 can include and/or beconfigured to interface with any or all of: a Picture Archiving andCommunication System (PACS) and/or alternative image viewing and imagestorage platform, a voice recognition platform, an intelligent radiologyworklist and/or alternative radiology worklist, a Radiology InformationSystem (RIS) and/or alternative patient tracking platform, an electronicmedical record (EMR) database, an electronic health record (EHR)database, a Clinical Information System (CIS) platform and/oralternative management software, a Health Information System (HIS)platform and/or alternative management software, a LaboratoryInformation System (LIS) platform and/or alternative managementsoftware, one or more vendor-neutral archive (VNA) components, and/orany other suitable components. The system 100 can optionally be tailoredto the preferences of a particular radiology group, tailored to thepreferences of multiple radiology groups, agnostic of radiology grouppreferences, tailored to the preferences of an individual radiologistand/or aggregated set of radiologists, and/or otherwise configured.

In a preferred set of variations, the system 100 is configured tointerface with (e.g., integrate with, communicate with, be built on topof, as a virtual machine, etc.) a radiology platform including a speechrecognition platform, wherein the method 200 is adapted to integratewith the features of the particular radiology platform (e.g., hotkeys,radiology report formatting, inclusion or removal of specific languagein the report dependent on adjustable configuration(s), varyingplacement of caret (e.g., cursor) or selection of text after generationof impression, variations in user interface, user notifications, optionsfor collecting user feedback, etc.).

In additional or alternative variations, the system 10 o includes any orall of the radiology platform and/or a corresponding method 200 can beotherwise suitably integrated with a radiology platform.

4. Method

As shown in FIG. 2 , a method 200 for automatically generatingimpression text (and/or any other suitable fields of a radiology reportsuch as comparisons, contrast amounts, specific measurements, etc.)includes: receiving a radiologist identifier (radiologist ID); receivinga set of finding inputs; determining a context of each of the set offinding inputs; determining an impression based on the context and theradiologist style; and inserting the impression text (and/or any othersuitable text) into the report. Additionally or alternatively, themethod 200 can include any or all of: determining a radiologist style,training a set of models, preprocessing information, postprocessinginformation, receiving other inputs (e.g., worklist, PACS information,RIS information, HIS information, EHR and/or EMR information, etc.),and/or any other suitable processes performed in any suitable order.

The method 200 functions to generate an automated impression from a setof radiology findings and within the style of a particular radiologist.Additionally or alternatively, the method 200 can function to generateany suitable portion or all of a radiology report, reduce the timeand/or effort required to generate a radiology report, learn aradiologist's particular style of writing and/or dictating impressionsand/or findings, increase and/or maintain accuracy of an impressionsand/or findings section, enable compliance with and/or optimization ofone or more healthcare facility and/or radiology organization standards,and/or perform any other suitable function(s).

An impression herein refers to an impression of a radiology report,which conventionally uses the findings (and/or any or all of the patientclinical history, indication for the imaging study, etc.) to summarize apatient condition (e.g., provide a diagnosis and/or summarize thefindings) and/or otherwise summarize the state of the patient (e.g.,recommend and/or prescribe a follow-up for the patient and/or aparticular disease state, posit one or more potential disease states,etc.). Because the impression section typically offers criticalinformation for decision-making, it is conventionally considered to bethe most important and/or personalized part of the radiology report.

The findings section conventionally lists the radiologist's observationsand findings regarding a set of one or more areas of the body examinedin an imaging study (e.g., computed tomography [CT] imaging, magneticresonance imaging [MRI], ultrasound imaging, radiography, etc.). Theradiologist can indicate whether each area was found to be normal,abnormal or potentially abnormal. Sometimes an area of the body isincluded and can be evaluated using the images, but is not discussed.This situation typically means that the radiologist did not find thearea noteworthy for comment.

The method 200 is preferably configured to be integrated within astandard radiology workflow, but can additionally or alternatively beconfigured to replace one or more processes in a standard radiologyworkflow, be performed independently of a standard radiology workflow,and/or be otherwise performed.

The method 200 is preferably performed with a set of models (e.g., asdescribed above) and/or any suitable neural network architecture. Theset of models are preferably trained such that the set of models jointlylearns how to structure an impression from a finding and how aradiologist style (e.g., as defined as a matrix of coefficients) wouldinteract with that impression, but can additionally or alternatively beotherwise configured.

Generating an impression is preferably fully automated but canadditionally or alternatively be partially manually generated (e.g.,require portions to be manually entered, enable edits to be manuallyentered, provide multiple options for generated impression languageand/or additional or alternative language that the user can optionallyselect or deselect, etc.), and/or be otherwise determined.

4.1 Method—Training the Set of Models S205

The method 200 can optionally include training the set of models S205,which functions to train any or all of the set of models 110 used todetermine the impression section.

The set of models can be trained with any or all of: supervisedlearning, semi-supervised learning, unsupervised learning, and/or anyother suitable training processes. Training the models can additionallyor alternatively include fine tuning one or more models (e.g.,pretrained models) with radiology report data. Further additionally oralternatively, the method 200 can implement already trained modelsand/or any other suitable models and/or algorithms.

The method 200 can additionally or alternatively include one or morepreprocessing processes, such as preprocessing information used to trainone or more models.

Preprocessing information can optionally include removing or modifyingone or more false positives from training data when training the set ofmodels (e.g., during initial training, during a second and/or iterativetraining, during a fine-tuning process, etc.), which can function to:make the system and/or method more robust in determining an accurateimpression, prevent and/or minimize machine learning hallucinations,and/or perform any other function(s). In some radiology reports, forinstance, the findings section does not include a particular findingwhich the impression section does include. In a specific example, aradiologist does not mention anything about the enlarged size of thepatient's heart in the findings section but then realizes it at the lastminute and includes it only in the impression section. This may alsoinclude removing or modifying one or more word or phrase repeats,comments not found within the findings, templated text, or macro textfrom the impression section. Removing or modifying false positives fromthe training data can function to help prevent the system from insertingthese “non-finding” impressions randomly and/or when notwarranted/relevant.

Additionally or alternatively, preprocessing can include removing ormodifying training data corresponding to false negatives, such asradiology reports in which a clinically significant finding is mentionedbut not commented on in the impression section, or reports in whichmeasurements and/or other language is swapped between or among two ormore distinct findings, and/or any other suitable data.

Preprocessing can optionally additionally or alternatively includeadding and/or removing items from training data (e.g., using asyntheticator), which can include any or all of: training the model(s)to add phrasing and/or use particular language which is recommendedand/or prescribed (e.g., based on standards of a radiology societyand/or group, based on preferences of a radiology group, based onpreferences of a particular radiologist, based on preferences of ahealthcare facility for coding and/or billing optimization and/or tosatisfy coding and/or billing requirements or standards, etc.); trainingthe model(s) to not use phrasing that is recommended against; add,remove, and/or edit formatting from radiology reports used as trainingdata (e.g., remove call reports, signatures, and/or other templatedtext); and/or add or remove any other items, words, phrases, orsentences (e.g., including any form of data perturbation). In a specificexample, for a particular radiology group, for instance, it may be goodpractice in cases of rib trauma with chest pain or tenderness to saywhether or not there are displaced rib fractures in a chest X-ray. Inthese cases, the method 200 can optionally include correcting traumachest X-ray training data for this radiology group and/or others whichdoes not mention this.

Preprocessing can optionally additionally or alternatively includeupweighting or upsampling sets of cases in the training data, which canfunction to help the model(s) properly handle complex and/orparticularly important (e.g., critical, life-threatening, etc.) cases.In some variations, for instance, the method 200 includes implementing aloss function to upweight cases so that model pays closer attention tothem. In specific examples, cancer cases, which are complicated tointerpret and determine accurate impressions for, are upweighted througha loss function. In other variations, for instance, the method 200includes upsampling of such complex cases.

Preprocessing can optionally additionally or alternatively includeimplementing subword vocabulary embeddings.

Preprocessing can optionally additionally or alternatively includetokenizing one or more recommendations, which functions to increase theprocessing speed. In some variations, for instance, preprocessingincludes replacing a relatively long string of text in the radiologyreport (e.g., in the impression section, in any section of the radiologyreport, etc.) with a particular token representing the replacement andindicating in post-processing that the token should be reverted back tothe original string, a variation of the original string, or theassociated or unassociated output from one or more models or otherpredictive modeling or algorithm(s) (e.g., machine learning algorithms)specific to one or more recommendations. This can additionally oralternatively performed with the models themselves (e.g., later in themethod 200), wherein one or more models are trained specifically to oneor more types of recommendations, which is used to select theappropriate consensus guideline recommendation as an output in anautomatically generated radiology impression. The language of theserecommendations can be manually or automatically updated as consensusguidelines change, and may also be modified by or for specific radiologygroups, specific health systems or hospitals, and/or individualradiologists.

Preprocessing can optionally additionally or alternatively includetraining separate models based on a set of preferences (e.g., preferredand/or prescribed recommendations, radiology group preferences,radiologist preferences, healthcare facility preferences, preferredfollow-up treatments, etc.). In some variations, for instance, modelsare trained separately to be able to determine particularrecommendations based on the patient's condition (e.g., as determined inthe findings). These recommendations can be any or all of: guided byradiologist advisors and/or societies; guided by publications and/orflowcharts; guided by collaborative care team decisions on follow-uptreatment; and/or any other suitable information. Additionally oralternatively, the models can be trained to reflect recommendationswhich often get updated. In some variations, for instance, therecommendation data being trained on is historical, wherein the methodincludes tagging that recommendation and updating it and/or flagging itso that it can be updated. In specific examples, a token is used to taga portion of the impression (e.g., corresponding to outdated and/orpotentially outdated information) when training the model, wherein inpost-processing, logic adjusts the language corresponding to the tokento reflect the up-to-date language.

The method 200 can additionally or alternatively include any otherpreprocessing or combination of preprocessing processes; be performed inabsence of preprocessing; include preprocessing at other times in themethod 200; and/or be otherwise performed.

The method 200 can optionally include any number of postprocessingprocesses (e.g., to postprocess data after being processed by the set ofmodels), such as any or all of those described above and/or any othersuitable processes. Alternatively, the method 200 can be performed inabsence of postprocessing.

In some variations, for instance, postprocessing can includeautomatically marking (e.g., highlighting, providing a notificationassociated with, etc.) and/or changing language in an automatically orpartially automatically generated section (e.g., impression field) ofthe radiology report. Additionally or alternatively, this can beperformed during processing with the model(s), during preprocessing,multiples times throughout the method, and/or at any suitable time(s).

This can function to achieve any or all of the functions described above(e.g., compliance with radiologist standards or preference, billingrequirements, etc.) and/or any other suitable function(s). In someexamples, for instance, the language used in one or more portions ofgenerated text (e.g., as part of an automatically generated impressionsection) can be automatically changed (e.g., at the time of generation,in postprocessing, etc.) and optionally mentioned in one or morenotifications to the radiologist (e.g., depending on the type of change)as to why the change was made (e.g., and requiring no further actionfrom the radiologist, to receive confirmation from the radiologist, toreceive an edit and/or a rejection of the change from the radiologist,etc.). Additionally or alternatively, the text can be automaticallyhighlighted and/or otherwise marked, wherein the highlighted portionincludes language suggested to the radiologist to be changed (e.g., tooptimize one or more of: billing, reimbursement, and compliance with aset of radiology standards), and optionally requiring an input from theradiologist to change.

Additionally or alternatively, S205 can include any other suitableprocesses.

4.2 Method—Determining a Radiologist Style S210

The method 200 optionally includes determining a radiologist style S210,which functions to characterize any or all of the preferences,tendencies, idiosyncrasies, writing styles, summary writing styles,grammar, and/or any other characteristics of the radiologist whenwriting a radiologist report (e.g., impression section). The radiologiststyle is preferably specific to a particular radiologist, but canadditionally or alternatively be specific to a particular radiologygroup (e.g., to conform to a set of report requirements established bythe radiology group), a particular healthcare facility, and/or any otherentity.

The radiologist style can be used to determine any or all of: a lengthof one or more sections of the radiology report (e.g., length of animpression section), a brevity of one or more section(s), a word flow, atype (e.g., formal versus informal, difficulty level, language, etc.) ofwords used in one or more sections of the radiology report, a subset ofwords typically chosen by the radiologist (e.g., a set of wordsroutinely chosen by the radiologist over their respective synonyms, aset of words having been chosen previously by the radiologist, etc.), anordering and/or prioritization of a set of findings, a summarization ofa set of multiple findings (e.g., order in which multiple findings arelisted, which findings are grouped into more general findings, whichfindings are included in the impression and which are not, etc.),pertinent negative and/or global negative language (e.g., language thatdescribes the lack of specific relevant positive findings and/or generalpositive findings), the conclusion(s) generated from a set of findingssuch as radiologist-specific requirements for predicting a patientcondition (e.g., radiologist only characterizes a spine curvature asscoliosis if it has an angle of 10 degrees or greater), differentialdiagnoses generated from a set of findings (e.g., indicating that any ofthese three patient conditions could result in this set of findings, andpotentially discussing or explaining why one or more of these patientconditions is considered more or less likely), and/or any other suitablefeature of the section(s) of the radiology report.

The radiologist style is preferably determined based at least in part ondata from radiology reports previously generated (e.g., manuallygenerated) by the radiologist. Additionally or alternatively, theradiologist style can be determined based on other radiologist inputs(e.g., collected in surveys, questionnaires, etc.), predicted orsynthetic data (e.g., synthetic radiology reports approved by theradiologist, etc.), radiologist metadata (e.g., demographic information,experience level, etc.), radiology group information, and/or any othersuitable information.

The radiologist style is preferably in the form of a mapping (e.g.,matrix, vector, auxiliary field of another matrix such as a set of wordembeddings, etc.) including a set of weights to be used in subsequentprocess(es) of the method to generate an impression and/or any othersuitable section(s) of a radiology report, but can additionally oralternatively include any other data in any suitable data format. Theradiologist style is preferably determined through deep learning, suchas through any or all of: a set of trained models, a set of algorithms(e.g., machine learning algorithms), a set of neural networks, and/orany other suitable deep learning infrastructure. Additionally oralternatively, the radiologist style can be determined manually and/orthrough any other suitable process(es).

S210 is preferably performed once for each radiologist during anonboarding of the radiologist into the system 100. Additionally oralternatively, S210 can be performed once a radiologist has manuallygenerated a predetermined number of reports, completed a predeterminednumber of studies, achieved a certain level of experience, and/or basedon any other milestone. Further additionally or alternatively, S210 canbe performed multiple times (e.g., every time the method 200 isperformed, when a radiologist style is updated, when training one ormore machine learning models, when using one or more radiologist editsmade to generated impressions as training data for reinforcementlearning, upon the generation of a new report, based on radiologistprompting, based on radiology group prompting, once a predeterminedamount of time has passed, at a predetermined frequency, etc.), and/orat any other time(s).

The radiologist style can be determined based on any or all of: a set ofmodels (e.g., machine learning models, any or all of the set of models110, etc.); one or more algorithms; manually; and/or based on any otherprocesses.

In a first variation, S210 includes determining a radiologist stylevector with a trained model based on a set of (e.g., hundreds of, atleast 100, etc.) radiology reports previously manually generated by theradiologist, wherein the radiologist style vector is subsequently usedin method to automatically generate impression text of a radiologistreport.

4.3 Method—Receiving a Radiologist Identifier (Radiologist ID) S220

The method 200 includes receiving (e.g., retrieving, referencing, etc.)a radiologist ID S220, which functions to enable an impression sectionof the radiologist report to be determined in accordance with theradiologist's reporting language style. Additionally or alternatively,the radiologist ID can function to enable any other suitable section(s)of the radiologist report to be generated.

The radiologist ID is preferably linked to a radiologist style asdescribed above (e.g., within the architecture of one or more models,through a lookup table, etc.) but can additionally or alternatively beotherwise associated with one or more radiologist styles. Theradiologist ID is preferably assigned to a radiologist during anonboarding process (e.g., as described above, and referenced based onthe name of the radiologist on the radiology report, etc.), but canadditionally or alternatively be assigned upon the radiologist joining aradiology group, healthcare facility staff, and/or based on any othersuitable trigger.

The radiologist ID is preferably received at a computing systemconfigured to generate a radiology report impression section, furtherpreferably a remote computing system, but can additionally oralternatively be received at any suitable computing system. Theradiologist ID can be automatically received upon the generation of aset of radiology images, upon the initiation of the generation of areport, upon the generation and/or receipt of a set of finding inputs,upon the completion of one or more sections of a radiology report, onceentered by a radiologist or other user, and/or at any suitable timebased on any suitable trigger(s).

S220 can optionally include receiving any other suitable information,such as any or all of: patient information (e.g., patient age, otherpatient demographic information, etc.), patient history, healthcarefacility information, and/or any other suitable information.

In a first variation, the radiologist ID is assigned to each radiologistupon being onboarded into the system 100, wherein the radiologist ID isassociated with a particular radiologist style vector, wherein theradiologist style vector is determined and/or received at a remotecomputing subsystem (e.g., from remote storage, from local storage,etc.).

4.4 Method—Receiving a Set of Finding Inputs S230

The method 200 includes receiving a set of finding inputs S230, which(e.g., along with the radiologist style) functions to receiveinformation associated with a set of radiology images with which animpression can be generated.

The set of finding inputs preferably includes the entire findingssection of a radiology report, wherein the findings section isrepresented as a string of text, and wherein each of the set of findinginputs is a word in the string of text. Any or all of the findingssection can be manually entered (e.g., typed) by a radiologist or otheruser, transcribed from radiologist speech (e.g., with a speechrecognition program), automatically generated, and/or otherwisedetermined.

The method 200 can optionally include modifying the set of findinginputs, such as preprocessing the radiology report and/or findingssection (e.g., removing formatting, preprocessing as described above,etc.) and/or performing any other suitable edits on the findings sectionand/or any other suitable sections of the radiology report.

The set of finding inputs are preferably received at a computing system(e.g., wherein the radiologist ID is received), further preferably at aremote computing system. The set of finding inputs can be received uponcompletion of the findings section (e.g., as determined based onradiologist input, as determined based on monitoring movement of a caretassociated with a radiologist display, etc.), prior to a findingssection being completed, and/or at any other suitable time(s).

In some variations, S230 includes determining when the findings sectionhas been completed, which functions to prevent an impression sectionfrom being prematurely generated (e.g., with incomplete information).Determining this can include any or all of: checking for empty fields inthe report, monitoring and/or checking for changes to the report (e.g.,checking for a change from a templated entry within a field to adictated, typed or macro-based entry within the field), or otherwisemonitoring the report.

In some variations, S230 includes determining when the impressionsection has been or is being dictated, typed, inserted as macro text, orotherwise manually added by the radiologist, which functions to preventan impression section from being generated with a zero-click generationprocess when not needed. Determining this can include any or all of:monitoring and/or checking for changes in the impression section(s)(e.g., checking for a change from a templated entry within a field to adictated, typed or macro-based entry within the field), or otherwisemonitoring the report.

In a first variation, S230 includes receiving at a remote computingsubsystem the set of words collectively making up a completed findingssection of the report, wherein an embedding process is performed tonumerically represent the set of words.

4.5 Method—Determining a Context of Each of the Set of Finding InputsS240

The method 200 includes determining a context of each of the set offinding inputs S240, which functions to enable the generation of anaccurate, intelligible radiology impression based on the set of findingsas a whole. Additionally or alternatively, S240 can function to generatean accurate, intelligible radiology impression based on any or all of:patient information, healthcare facility information, and/or any othersuitable information inputs. Further additionally or alternatively, S240can function to generate any suitable section(s) of the radiology reportin an accurate, intelligible, and/or radiologist-specific manner.

The context defines the relationship of each of the set of findinginputs with the other finding inputs (e.g., surrounding findings inputs,adjacent finding inputs, all other finding inputs, etc.) of the findingssection. The context of each finding input (e.g., word) can take intoaccount any or all of: a tense of other finding inputs; a grammaticalcharacterization of other finding inputs (e.g., noun, verb, adjective,adverb, etc.); a subject matter characterization of other finding inputs(e.g., technical word, medical word, linking word, patient conditioncharacterization, parameter value, etc.); an ontology of any or all ofthe finding inputs; a length of other finding inputs; the presence of anidentifier and/or signifier finding input; and/or any other feature(s)of the other finding inputs and/or the finding input itself.

The context is preferably determined with a set of models no asdescribed above, further preferably with a model including one or moreattention processes (e.g., self-attention process, during an encodingprocess, with a self-attention layer in an encoder, with aself-attention layer in a decoder, during an encoder-decoder attentionprocess, etc.), such as, but not limited to, any or all of the attentionprocesses described above. One or more of the attention processespreferably assesses (e.g., quantifies, calculates, etc.) therelationship of each of a set of finding inputs, such as each word ofthe text of the radiology report's findings section, to each of theother finding inputs. This can function to determine how much eachfinding input depends on the other finding inputs in the overall set offinding inputs (e.g., collectively forming a sentence of the findingssection, collectively forming the entire findings section, etc.).Additionally or alternatively, this can function to enable thedetermination of an impression which takes into account multipledifferent findings (e.g., recommendation for a patient experiencingcomorbidities); enable the ranking and/or relative importance of one ormore findings, and/or perform any other suitable functions.

A set of encoders (e.g., encoding layers) are used to implement the oneor more attention processes, wherein the encoders function to determinea context matrix (e.g., vector, 2-dimensional matrix, 3-dimensionalmatrix, etc.) associated with the finding inputs. Each of the encoderspreferably includes a self-attention mechanism (e.g., self-attentionlayer) and a feed forward neural network, but can additionally oralternatively include any suitable architecture with any suitable neuralnetworks (e.g., convolutional, recursive, recurrent, etc.) and/or layers(e.g., attention layers). Any or all of the encoders preferably receiveas an input a set of word embedding vectors and/or one or more wordembedding matrices (e.g., as determined with an embedding algorithm, asdetermined with the first encoder, a 512-dimensional word embeddingvector for each word, etc.), which characterizes the text of thefindings section. Additionally or alternatively, any or all of theencoders can receive any other representation of the set of findingsinputs (e.g., text itself) and/or any other suitable information asinputs. In some variations, for instance, the matrix includes one ormore auxiliary fields, such as any or all of: the radiologist stylematrix, other radiologist information, patient information, radiologygroup and/or healthcare facility information, and/or any other suitableinformation. In specific examples, for instance, the radiologist stylematrix is concatenated with a word embeddings matrix and/or each of aset of word embedding vectors and processed by a set of encoders todetermine the context matrix. Further additionally or alternatively, anyor all of this information can be received separately (e.g., in anothermatrix); received at another time in the method 200 (e.g., radiologiststyle matrix is concatenated with the context matrix later in the method200); received at multiple processes in the method; not received; and/orotherwise received.

The set of encoders preferably additionally receives and/or determines aset of positional encodings, which functions to enable the order of thewords in the findings section to be taken into account for determiningcontext. In variations involving a transformer model, for instance, inwhich processing of the finding inputs is performed at least partiallyin parallel, a positional encoding associated with each of the findinginputs is preferably used to enable order (e.g., position of the findinginput within the findings section) to be taken into account whendetermining context.

S240 can optionally include running an information extractor over thefinding inputs and using the results as input to the set of encoders(e.g., as an auxiliary field, with the set of word embeddings, with theset of word embeddings and positional encoder, etc.). The informationextractor preferably functions to determine any concepts associated withthe set of finding inputs, such as concepts as determined by one or moreontology databases (e.g., clinical ontology database). In a first set ofvariations, for instance, an ontology database is used to determine aset of concept identifiers [IDs] associated with the set of findinginputs (e.g., a subset of the finding inputs), wherein the conceptidentifiers can be used as any or all of: a separate input (e.g., table,matrix, vector, etc.) relative to the word embeddings; an input with theword embeddings (e.g., concatenated with the word embeddings and/or aword embedding matrix); and/or otherwise received as an input to a setof encoders. Additionally or alternatively, an ontology databased can beused during a training of one or more models. In some variations, forinstance, associating words of the impression fields of training datawith concepts in the ontology databased can be used to better train themodels, be used in inference, and/or be used in any other suitable ways.

Extracting information to be used in S240 can optionally includedetecting negation, which refers to findings that indicate something(e.g., a condition) is not there. For instance, a finding whichindicates that the patient's heart is “not enlarged,” the negation “not”can be indicated in and/or associated with the relevant wordembedding(s). In some examples, an auxiliary field is added and/orassociated with the word embeddings to indicate which finding inputs areassociated with a negation, thereby increasing an accuracy of theresulting impression. In examples further including extractinginformation from an ontology database, this can help look forappropriate concepts (e.g., normal sized heart concepts). Additionallyor alternatively, negation can be otherwise implemented and/or S240 caninclude any other suitable processes and/or absence of a negationdetection process.

In a set of specific examples, S240 includes determining a set ofconcept IDs, which are words, associated with particular finding inputswith an ontology database and determining embeddings for the concept IDsin a separate embedding table and/or matrix and/or vector; adding atoken type to the word embeddings and the concept ID embeddings (e.g., atoken type of “0” for words and a token type of “1” for concepts); andconcatenating the word embeddings and the concept embeddings. Theconcatenated matrix further preferably includes and/or is determinedbased on positional encodings of the embeddings, so that the findingword associated with the particular concept are aligned. Additionally oralternatively, determining and using the concept IDs can be otherwiseperformed.

Additionally or alternatively, the information extractor can determineany other suitable information.

S240 preferably produces one or more of the following as an output: asingle context matrix (e.g., context vector, matrix formed from multiplecontext vectors, matrix formed from multiple context matrices, etc.);multiple context matrices (e.g., set of multiple context vectors, acontext matrix for each finding input/word embedding); any othermatrices and/or parameter values (e.g., confidence score); and/or anycombination of matrices and/or values. Additionally or alternatively,S240 can produce output relating and/or corresponding to concepts fromone or more ontology databases (e.g., clinical ontology database).Additionally or alternatively, S240 can produce any other suitableoutputs and/or combination of outputs.

In some variations, for instance, the context of each finding input isrepresented in a matrix or vector which includes a set of valuesindicating the dependence of the finding input on each of a set ofsurrounding finding inputs. The surrounding findings inputs cancollectively include the other words in the sentence in which thefinding input word is placed, all other words in the findings section,and/or any other suitable words in any suitable section of the radiologyreport.

Additionally or alternatively, determining the context can include anyother suitable processes, such as any or all of: eliminating findinginputs (e.g., based on low dependency of other finding inputs on thespecific finding input, based on the determination of a redundantfinding input, etc.), referencing other sections of the report,referencing the radiologist style, updating and/or otherwise alteringthe radiologist style, and/or otherwise determining the context of thefinding inputs.

In a first variation, a context matrix is determined with a set ofencoders (e.g., 6 encoders, 7 encoders, 8 encoders, 9 encoders, 10encoders, 11 encoders, 12 encoders, between 12 and 15 encoders, between5 and 10 encoders, between 10 and 15 encoders, greater than 15 encoders,etc.) in an encoder-decoder architecture of a transformer model. Inspecific examples, determining a context of each of the set of findinginputs includes, at self-attention layers of the encoders, calculating aset of context values for each finding input, wherein set of contextvalues quantifies how much the finding input depends on each of thesurrounding finding inputs. The context matrix is preferably determinedbased on a set of word embeddings, each of the set of word embeddingsassociated with a word of a findings section. The word embeddings canoptionally be concatenated with and/or otherwise associated with (e.g.,in a separate matrix) other information (e.g., forming auxiliary fieldsof a concatenated matrix), such as, but not limited to, any or all of:concepts associated with any words of the findings section (e.g., asdetermined using an ontology database); positional encodings; negations;and/or any other suitable information.

4.6 Method—Determining an Impression Based on the Context and theRadiologist Style S250

The method 200 includes determining an impression based on the contextand the radiologist style S250, which functions to automatically fill inan impression field of a radiology report. Additionally oralternatively, S250 can function to mimic the particular style (e.g.,length, level of detail, word choice, content of recommendations and/orfollow-up steps, etc.) of a radiologist; determine an accurateimpression based on a set of findings; enable the determination oftunable (e.g., in length, detail, time required to generate, etc.)impression field; and/or perform any other function(s).

The impression (which equivalently refers to herein any or all of: asingle impression, a sentence of an impression section, the entireimpression section, and/or any other suitable portion and/or feature ofthe impression section of a radiology report) is preferably generated ona word-by-word basis through a set of one or more decoding processes,wherein a set of the decoders determine each word of the impression. Theset of decoders preferably determines each word by selecting it from aset of potential words based on a probability score assigned to each ofthe set of potential words. The probability score is further preferablycalculated based on the context values (e.g., in the form of a set ofcontext value vectors, in the form of a context value matrix, etc.)associated with each of the set of finding inputs, along with the set ofradiologist style coefficients (e.g., in the form of a radiologist stylevector, in the form of a radiologist style matrix, etc.).

S250 preferably includes concatenating the radiologist style matrix(e.g., radiologist style vector) with the context matrix, wherein theconcatenated matrix is processed in S250 (e.g., in a set of decoders asdescribed below). The context values and the radiologist style valuescan be concatenated together in any or all of the following: into asingle vector for each finding input, into a matrix for each findinginput, into a matrix for all finding inputs, and/or otherwise combined.Additionally or alternatively, the radiologist style matrix and thecontext matrix can do any or all of: remain in separate vectors and/ormatrices, be used in the determination of a new vector or matrix, and/orbe otherwise associated or independent.

The set of potential words for each word entry in the impression fieldis preferably determined based on a set of neural networks (e.g., feedforward neural networks) within a set of decoders (e.g., decoderlayers). The decoders are preferably part of an encoder-decoderarchitecture of a transformer (e.g., multi-transformer) model, but canadditionally or alternatively be integrated within any suitablemodel(s). The set of decoders can include any suitable number of layers(equivalently referred to herein as number of decoders), such as any orall of: between 1 and 6 (e.g., 1, 2, 3, 4, 5, 6), between 6 and 12(e.g., 6, 7, 8, 9, 10, 11, 12), greater than 12 (e.g., 13, 14, 15, 16,etc.), between 15-20 (e.g., 15, 16, 17, 18, 19, 20), greater than 20(e.g., 25, 30, 35, etc.), and/or any suitable number. In somevariations, the number of decoders or number of layers within a decoderis reduced (e.g., relative to conventional decoders) to speed up theprocessing (e.g., a beam search process as described below) of thedecoders. In specific examples, for instance, a conventional 12-layerdecoder is reduced to 6 layers to speed up a beam search process.

The set of decoders can optionally include an odd number of layers(e.g., in a standard attention process of a transformer model).Alternatively, the set of decoders can optionally be even, which canfunction, for instance, to enable global context attention. In specificexamples, this can lead to more accurate results.

Each of the decoders in the set of decoders preferably includes aself-attention mechanism (e.g., self-attention layer), an attentionmechanism (e.g., attention layer over the encodings), and a feed forwardneural network. Additionally or alternatively, any or all of thedecoders can have any suitable architecture including any number andtype of layers, any neural network (e.g., convolutional, recursive,recurrent, etc.) and/or combination of neural networks, an absence ofone or more layers and/or neural networks, and/or any combination ofcomponents.

S250 can additionally or alternatively include any number of additionalprocesses, such as an encoding process, a normalization process, and/orany other suitable process(es).

The impression section is preferably determined on a word-by-word basis,such as one word at a time in a sequential fashion. Additionally oralternatively, any or all of the words can be determined in paralleland/or in other non-sequential fashion; a single pass decoding (e.g.,non-auto-regressive decoding) process can be implemented to decode thewhole sequence and/or a portion of the sequence at one time (e.g., toincrease the speed of processing; and/or the impression text can bedetermined in any suitable way and/or time(s).

Determining the impression preferably includes determining and assigningprobability scores (e.g., with the neural networks of the decoders, witha separate neural network, with a separate feed forward neural network,etc.) to potential words of the impression (e.g., generating theimpression text one word at a time), wherein the selected words areassociated with the highest probability scores (e.g., relative to theother words being considered for that position in the impression text).Additionally or alternatively, the impression can be any or all of:generated in absence of probability scores (e.g., with a ranking and/orprioritization of words); generated with manual input (e.g., from aradiologist); generated with random selection of one or more words;and/or generated in any other suitable way(s).

In a preferred set of variations, probability scores are determinedand/or used in accordance with a beam search decoding process todetermine the text of the impression. The beam search decoder ispreferably associated with and/or assigned a particular beam width value(e.g., 2, 3, 4, 5, 6, 7, 8, 9, 10, between 2 and 5, between 5 and 10,greater than 10, etc.), wherein the beam width corresponds to (e.g., isequal to, is related to, etc.) the number of words considered at eachposition. Additionally or alternatively, any other suitable process canbe used (e.g., Greedy Search algorithm) in selecting the words for theimpression text.

In a specific example, for instance, a beam width of the beam searchprocess is equal to three, wherein each word of the impression isselected from a set of three potential words, each assigned aprobability score based on the set of context values and the radiologiststyle values. The output of the beam search process is preferably a mostprobable sequence of words, which forms the impression text. Determiningthe most probable sequence preferably includes routinely terminatingleast probable paths based on the probability scores determined at eachpotential word calculation, but can additionally or alternativelyinclude any other processes (e.g., elimination processes). Furtheradditionally or alternatively, the beam width can have any suitablevalue (e.g., less than three, greater than three, etc.).

S250 can optionally include performing a length prediction process,which functions to predict the length that the impression would be(e.g., if manually generated by the particular radiologist, based on anaggregated set of radiologists, based on the findings, etc.), andproduce an impression based on the predicted length. In decodingprocesses, for instance, the length of the output can be chosen,designed, and/or tuned. In some variations, for instance, a training ofthe set of models includes learning how long an impression generally isbased on the length of the impression field in the training data (e.g.,manually generated radiology reports, synthetic radiology reports,etc.), which can then be used to prescribe the length of theautomatically generated impression. A length prediction module used toperform the length prediction can be any or all of: part of the modelperforming the decoding; part of a separate model (e.g., separate thanthe model performing the encoding and/or decoding, separate from atransformer model, etc.); and/or part of any suitable models. The lengthprediction module can receive a radiologist ID and/or radiologist styleas an input. Additionally or alternatively, the length prediction modulecan receive any other suitable input(s).

In some variations of S250, one or more parameters of the impressionfield are tunable, such as any or all of: length, level of detail,comprehensiveness (e.g., number of findings commented upon), and/or anyother suitable parameters. In some examples, for instance, a parameterassociated with a length of the impression section (e.g., quantifyingthe length, number of words, level of detail, etc.) is tunable by any orall of: a particular radiologist, a radiology group, a healthcarefacility, and/or any other suitable entities or combination of entities.In specific examples, a parameter alpha, which is an exponent thatweights the scores returned out of a beam search decoding process, isable to be adjusted by a radiologist or by a radiology group, wherein asmaller value of alpha returns an impression faster but at a shorterlength and/or lesser level of detail, whereas a larger value of alphareturns an impression slower but at a longer length and/or greater levelof detail. Additionally or alternatively, any other suitable parameterscan be tunable at any suitable processes of the method (e.g., in S240).

In a first variation of S250, S250 includes performing a decodingprocess of the outputs determined in S240 (e.g., with one or moreencoders of S240), wherein the decoding process includes a beam searchprocess performed with a set of decoder layers (e.g., 6, 12, etc.),wherein the impression text is generated on a word-by-word basis basedon a probability associated with each of a set of word options asprescribed by the beam width of the beam search decoder(s), theprobabilities determined based on one or more context matrices and oneor more radiologist style matrices.

4.7 Method—Inserting the Impression into the Report S260

The method 200 includes inserting the impression into the radiologyreport S260, which functions to complete the impression section of theradiology report. Additionally or alternatively, S260 can function toperform any or all of: concluding (e.g., automatically concluding) thegeneration of a radiology report; triggering a next action (e.g.,uploading the radiology report to a database, transmitting the radiologyreport to another entity such as a physician treating the patient and/ora clinical care team, notifying the radiologist that the report has beencompleted, prompting a radiologist to review and/or edit the impressionsection, etc.); requiring zero clicks from the radiologist to insert theimpression text; establishing integration with one or more radiologysoftware platforms; and/or performing any other suitable function(s).

S260 is preferably performed with zero clicks (e.g., no requiredhotkeys, no input from the radiologist, no mouse clicks, no keyboardclicks, etc.), but can additionally or alternatively be performed withone click, multiple clicks, the press of keyboard hotkey(s) or button ona dictation or navigation device, and/or based on any other input(s)and/or trigger(s) (e.g., from a radiologist). Insertion can be performedwith and/or include any or all of: injection; user interface automation;copy and paste; and/or any other suitable processes. The way in whichthe impression is inserted is preferably determined based on theparticular software program and/or particular radiology platform beingimplemented. Additionally or alternatively, an impression can beotherwise integrated within the radiology report and/or require anynumber of inputs and/or clicks (e.g., single click).

S260 can include any number of processes and/or detection of featuresfor determining the appropriate location at which to insert theimpression text (e.g., to insert the text in the proper location, toprevent overriding of other sections, etc.), since this location (e.g.,and type of box such as RTF edit box, etc.) can vary depending on theradiology platform being used (e.g., Fluency vs. PowerScribe). Theseprocesses preferably include invisibly monitoring the location of acaret (e.g., cursor) within the radiology report document, anddetermining the appropriate location of the impression field based onnavigation of the caret (e.g., as shown in FIGS. 3A-3D). In somevariations, for instance, the impression section is detected based ondetecting one of a variety of impression headers, and by characterizingthe text that follows the impression section (e.g., a callreport/critical value sentence; a different section header in the caseof reports with different arrangements of findings vs. impressions,etc.), and determining where the caret is in relation to these headersand other text.

In additional or alternative variations, it is known that the impressionfield is usually located at the end of the report, so a determinationthat there is a header denoting the impression section prior to thecaret and no text in front of the caret indicates that the caret islikely in the impression field, and can thereby trigger the insertion ofimpression text.

The trigger for entering the impression text in the impression field caninclude detecting that a caret has entered into the impression field;additionally or alternatively, the impression text can be inserted basedon a different trigger, based on multiple triggers, in absence of atrigger, otherwise, and/or based on any combination of these.

A variation of the information used to train the set of models, processthe set of models, determine impression text, and insert impression textis shown in FIG. 5 .

5. Variations

In a first variation of the system 100 (e.g., as shown in FIG. 4 ), thesystem includes: a set of models 110, wherein the set of models includesa transformer model including a set of encoders and set of decoders, theset of decoders including one or more beam search decoders, wherein theset of encoders receives the radiologist style matrix and the set offindings from the findings section of the radiologist report todetermine a context matrix, which is used by the beam search decoder toautomatically generate the impression section. The system interfaceswith a radiology platform including a speech recognition platform,wherein the speech recognition platform can be used to provide thefindings section to the system, receive the automated impression andinsert it into the report, and/or be otherwise integrated with and/or incommunication with the system. The system 100 can additionally includeany other suitable components, such as any or all of: a preprocessingmodule (e.g., as described above), a post processing module (e.g., asdescribed above), a computing system, and/or any other suitablecomponents and/or combination of components.

In a first variation of the method 200, the method includes any or allof: during an onboarding process of a radiologist, determining aradiologist style matrix (e.g., vector) based on a set of multiple(e.g., at least 100; at least 300; at least 1,000; at least 10,000;between 1,000 and 100,000; at least 100,000; at least 200,000; etc.)manually-generated radiology reports completed by the radiologist,wherein the radiologist style matrix includes a set coefficientsdefining the radiologist's style (e.g., word choice, word flow, length,summary of multiple findings, prioritization of findings, etc.) ingenerating impression text of the radiologist report; upon a triggerindicating that a report is to be generated and/or is in the process ofbeing generated by the radiologist, receiving a radiologist ID at aremote computing system (e.g., upon initiation of the associatedradiology report, upon detecting that a subset of sections of the reporthave been completed, upon detecting that the findings section of thereport has been completed, upon input from a radiologist, etc.), whereinthe radiologist style matrix is associated with the radiologist ID(e.g., within the architecture of the models, in a cloud-based lookuptable, etc.); receiving a set of finding inputs from a findings sectionof the radiologist report (e.g., upon detecting that the findingssection has been completed, upon detecting that all sections besides theimpression section have been completed, upon receipt of the radiologistID, etc.) at the remote computing system, wherein each of the findinginputs is an identifier (e.g., numeric identifier) corresponding to aword of the string of words making up the findings section; determininga set of one or more context matrices associated with the finding inputs(e.g., a context matrix for each finding input) with a set of one ormore attention process(es) (e.g., attention layer in an encoder,attention layer in a decoder, etc.), wherein the context matrix definesthe relationship between the finding input and the other finding inputsof the collective findings section; determining a set of first wordoptions and associated probabilities based on the radiologist style andthe context (e.g., in a concatenated matrix, in a concatenated vector,etc.); repeating this for subsequent words; generating a string of wordsfor the impression text with a beam search process (e.g., beam searchdecoder) based on these probabilities; and inserting the impression textinto the radiology report (e.g., based on detecting one of a variety ofimpression headers, characterizing the text that follows the impressionsection, and determining where the caret is in relation to these headersand other text; based on a trigger indicating that the radiologist hasentered the impression section and that a cursor is arranged in a fieldwith no subsequent text; etc.). The method can optionally additionallyinclude receiving patient information (e.g., with the radiologist ID),such as a patient age; receiving healthcare facility and/or radiologistorganization information (e.g., with the radiologist ID), such as anorganization code; and/or receiving any other metadata associated withgenerating a radiology report; performing pre-processing of any or allof the input(s); and/or any other process(es).

Additionally or alternatively, the method 200 can include any or all of:training the set of models (e.g., based on manually generated radiologyreports, based on synthetic radiology reports, based on the informationshown in FIG. 5 , etc.); performing one or more preprocessing processes(e.g., removing or modifying formatting, false positives, and/or falsenegatives for the radiology reports being used in training); performingone or more postprocessing processes (e.g., checking the impression textfor words in compliance and/or not in compliance with one or morestandards, alerting the radiologist to words or phrases not incompliance with one or standards, replacing words or phrases not incompliance with one or more standards, etc.); receiving an inputregarding a tunable parameter (e.g., length of impression section, levelof detail, etc.) from the radiologist and adjusting the impressionsection (e.g., generating a new impression section) based on theparameter value; referencing an ontology database to determine conceptassociated with the set of findings; embedding the concepts as auxiliaryfields in a matrix including word embeddings associated with thefindings, wherein the concepts and findings are linked based onpositional encodings; and/or any other suitable processes performed inany suitable order.

Although omitted for conciseness, the preferred embodiments includeevery combination and permutation of the various system components andthe various method processes, wherein the method processes can beperformed in any suitable order, sequentially or concurrently.

Embodiments of the system and/or method can include every combinationand permutation of the various system components and the various methodprocesses, wherein one or more instances of the method and/or processesdescribed herein can be performed asynchronously (e.g., sequentially),contemporaneously (e.g., concurrently, in parallel, etc.), or in anyother suitable order by and/or using one or more instances of thesystems, elements, and/or entities described herein. Components and/orprocesses of the following system and/or method can be used with, inaddition to, in lieu of, or otherwise integrated with all or a portionof the systems and/or methods disclosed in the applications mentionedabove, each of which are incorporated in their entirety by thisreference.

As a person skilled in the art will recognize from the previous detaileddescription and from the figures and claims, modifications and changescan be made to the preferred embodiments of the invention withoutdeparting from the scope of this invention defined in the followingclaims.

We claim:
 1. A method for automatically generating an impression sectionof a radiology report, the method comprising: receiving: a radiologistidentifier associated with a radiologist; and a set of finding inputs;with a trained machine learning model comprising a set of encoders and aset of decoders: determining a radiologist style matrix based on theradiologist identifier, wherein the radiologist style matrix isdetermined based on a length metric associated with a set of impressionsections in a set of manually generated radiology reports, the set ofmanually generated radiology reports generated by the radiologist;referencing an ontology database to determine a set of conceptsassociated with the set of finding inputs; determining, with the set ofencoders, a context matrix based on the set of finding inputs and theset of concepts; generating, with the set of decoders, the impressionsection of the radiology report based on the context matrix and theradiologist style matrix, wherein the impression section is configuredto mimic a writing style of the radiologist; retraining the trainedmachine learning model based on the generated impression section of theradiology report to determine a retrained machine learning model; andwith the retrained machine learning model, automatically generating asecond impression section of a second radiology report associated withthe radiologist.
 2. The method of claim 1, wherein the trained machinelearning model comprises a trained machine learning transformer model.3. The method of claim 2, wherein the trained machine learningtransformer model comprises the set of encoders and the set of decoders.4. The method of claim 1, wherein referencing the ontology databasecomprises comparing at least a subset of the set of finding inputs withthe ontology database to determine the set of concepts.
 5. The method ofclaim 4, wherein the ontology database is determined at least in partbased on a set of impression outputs, the set of impression outputsdetermined based on the set of manually generated radiology reports. 6.The method of claim 1, wherein the impression section comprises a stringof text, wherein the string of text is generated on a word-by-wordbasis.
 7. The method of claim 6, wherein each of a set of words of thestring of text is selected based on a probability associated with theword.
 8. The method of claim 6, wherein generating the string of text ona word-by-word basis is performed with a beam search decoder.
 9. Themethod of claim 1, wherein the radiologist style matrix is furtherdetermined based on a recommendation type associated with the set ofmanually generated radiology reports.
 10. The method of claim 1, whereinthe length metric comprises a number of words.
 11. The method of claim1, wherein generating the impression section of the radiology reportcomprises automatically inserting the generated impression section witha zero-click insertion process.
 12. The method of claim 11, whereinautomatically inserting the generated impression section comprisesautomatically determining a location for the impression section withinthe radiology report.
 13. The method of claim 12, wherein the locationis determined at least in part based on a monitored location of acursor.
 14. The method of claim 1, further comprising concatenating theradiologist style matrix and the context matrix to form a concatenatedmatrix, wherein the impression section is generated based on theconcatenated matrix.
 15. The method of claim 1, further comprisingproviding a tunable parameter at a radiologist interface, wherein theimpression section is automatically adjusted based on the tunableparameter.
 16. The method of claim 15, wherein the tunable parametercomprises at least one of a level of detail associated with theimpression section and a length associated with the impression section.17. The method of claim 16, wherein the retrained machine learning modelis determined based on the adjusted impression section.
 18. The methodof claim 1, wherein the set of finding inputs are associated with afirst token type, and the set of concepts are associated with a secondtoken type.